Using touch to localize flexible materials during manipulation

Localizing and manipulating features such as buttons, snaps, or grommets in fabrics and other flexible materials is a difficult robotics problem. Approaches that rely too much on sensing and localization that occurs before touching the material are likely to fail because the flexible material can move when the robot actually makes contact. This paper experimentally explores the possibility of using proprioceptive and load-based tactile information to localize a feature in the material while a robot is manipulating it. In our experiments, Robonaut 2, a robot with human-like hands and arms, interacts with features embedded in flexible materials that can be localized through touch. Particle filtering is used to localize the features that the robot touches based on proprioceptive and tactile measurements. We carefully evaluate the localization performance that can be expected when using proprioceptive information alone and the additional performance possible when tactile data is also available. The two types of measurements are shown to contain complementary information. We also show that load-based tactile measurements are a multi-modal function of state. A mixture of Gaussians measurement model is proposed that is shown to perform significantly better than a single Gaussian model.